On the quantitative assessment of predictive biomarkers.

Predictive biomarkers are among the most direct steps towards converting precision medicine into a clinical reality. By their very definition they are fundamentally different from prognostic or diagnostic biomarkers—instead of helping identify individuals who will develop the disease, they focus on differentiating between those who will and will not " benefit " from treatment. Given this difference, a natural question is whether the performance met-rics of sensitivity, specificity, positive predictive values (PPVs), and negative predictive values (NPVs) (" four metrics ") used in the prog-nostic/diagnostic context should be extended to the predictive setting. The two brief communications published in this issue of the Journal approach the question from two different perspectives. Using methods and assumptions similar to those adopted by Zhang et al. (1), Simon (2) presents simple formulas for estimating the four metrics, where the goal is predicting " benefit " at the individual level. That is, the interest is identifying individuals who would have improved outcomes with treatment but not without treatment and conversely individuals who would not have improved outcomes with treatment but not without treatment. Such metrics provide a clear understanding of the impact of treatment at the individual level. However, they suffer from inherent limitations owing to their reliance on unverifiable assumptions, arising from the fact that only one of two potential outcomes (one with treatment, one without treatment) can be observed on an individual. Simon argues that such analyses may still be useful without such assumptions if one employs a second " pragmatic interpretation. " Specifically, Simon shows that they can be interpreted as a comparison based on two randomly selected individuals, one treated and one control/standard of care with the same measured covariates. This interpretation circumvents the main criticism about estimating these quantities raised in the other communication by Janes et al. (3). This second interpretation may be of interest in evaluating the clinical utility of a biomarker at the population level and may in principle be estimated without any assumptions. Given the substantive concerns in Janes et al., the question arises as to whether or not there is a role for the " probability of treatment benefit " in assessing the predictive value of biomarkers in treatment decision-making. As an individual-level quantity, it has a deeper causal interpretation than the pragmatic population-level interpretation of Simon. However, as noted in Janes et al., restrictive assumptions are needed. This tradeoff is common in causal …